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(54) Method of image coding and decoding 

(57) A method of simplifying the arithmetic opera- 
tion in a global motion compensation process approxi- 
mates the motion vector field of the whole image without 
using many parameters. Motion vectors in the global 
motion compensation are found by the interpolation 
and/or extrapolation of the motion vectors of a plurality 
of representative points 602, 603 and 604 having partic- 
ular features in the spatial distance thereof. Since a shift 
operation can substitute the division for synthesizing a 
predicted image of global motion compensation, the 
processing using a computer or a dedicated hardware is 
simplified. 
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Description 

Field of the Invention 

5 The present invention relates to a method of coding and decoding an image by applying global motion compensa- 

tion to the whole image based on linear interpolation and/or extrapolation or bilinear interpolation and/or extrapolation. 

Backgroun d of the Invention 

10 In the highly efficient coding of a dynamic image, it has been known, in recognition of the similarity of the frames 
that are close to each other in regard to time, to use motion compensation in compressing the data. The most widely 
used motion compensation system at present image coding technology is block matching, employed in Standards 
H.261 . MPEG1 and MPEG2 which are international standards for a dynamic image coding system. According to this 
system, the image to be coded is divided into a number of blocks, and a motion vector is found for each of the blocks. 

75 Fig. 1 illustrates the constitution of a coder 100 of the H.261 Standard which employs a hybrid coding system 
(adaptive interframe/intraframe coding method) which is a combination of block matching and DCT (discrete cosine 
transform). A subtractor 102 calculates the difference between an input image (orignal image of present frame) 101 
and an output image 113 (that will be described later) of an interframe/intraframe switching unit 1 19. and outputs an 
error image 103. The error image is transformed into a DCT coefficient through a DCT processor 104 and is quantized 

20 through a quantizer 1 05 to obtain a quantized DCT coefficient 1 06. The quantized DCT coefficient is output as transfer 
data onto a communication line and is. at the same time, used in the coder to synthesize an interframe predicted image. 
A procedure tor synthesizing the predicted image will be described below. The quantized DCT coefficient 106 passes 
through a dequantizer 108 and an inverse DCT processor 109 to form a reconstructed error image 110 (the same 
image as the error image reproduced on the receiving side). 

25 An output image 1 1 3 (that will be descrfoed later) of the interframe/intraframe switching unit 1 1 9 is added thereto 
through an adder 111. thereby to obtain a reconstructed image 1 12 of the present frame (the same image as the recon- 
structed image of the present frame reproduced on the receiving side). The image is temporarily stored in a frame mem- 
ory 1 14 and is delayed in time by one frame. At the present moment, therefore, the frame memory 1 14 is outputting a 
reconstructed image 1 15 of the preceding frame. The reconstructed image of the preceding frame and the input image 

30 101 of the present frame are input to a block matching unit 1 1 6 where block matching is executed. 

In the block matching, an image is divided into a plurality of blocks, and a portion most resembling the original 
image of the present frame is taken out for each of the blocks from the reconstructed image of the preceding frame, 
thereby synthesizing a predicted image 1 1 7 of the present frame. At this moment, it is necessary to execute a process- 
ing (local motion estimation) for detecting how much the blocks have moved from the preceding frame to the present 

35 frame. The motion vectors of the blocks detected by the motion estimation are transmitted to the receiving side as 
motion data 1 20. From the motion data and the reconstructed image of the preceding frame, the receiving side can syn- 
thesize an estimated image which is the same as the one that is obtained independently on the transmitting side. 

Referring again to Fig. 1. the estimated image 117 is input together with a "0" signal 118 to the interframe/intra- 
frame switching unit 1 19. Upon selecting either of the two inputs, the switching unit switches the coding either the inter- 

40 frame coding or the intraframe coding. When the predicted image 117 is selected (Fig. 2 illustrates this case), the 
interframe coding is executed. When the "0" signal is selected, on the other hand, the input image is directly DCT-coded 
and is output to the communication line. Therefore, the intraframe coding is executed. 

In order to properly obtain the reconstructed image on the receiving side, it becomes necessary to know whether 
the interframe coding is executed or the intraframe coding is executed on the transmitting side. For this purpose, a dis- 

45 tinction flag 121 is output to the communication line. The final H.261 coded bit stream 123 is obtained by multiplexing 
the quantized DCT coefficient, motion vector, and interframe/intraframe distinction flag into multiplexed data in a multi- 
plexer 122. 

Fig. 2 illustrates the constitution of a decoder 200 for receiving a coded bit stream output from the coder of Fig. 1. 
The H.261 bit stream 217 that is received is separated through a separator 216 into a quantized DCT coefficient 201, 
so a motion vector 202, and an irrtraframe/interframe distinction flag 203. The quantized DCT coefficient 201 is decoded 
into an error image 206 through a dequantizer 204 and an inverse DCT processor 205. To the error image is added an 
output image 215 of an interframe/intraframe switching unit 214 through an adder 207 to form a reconstructed image 
208. 

The interframe/intraframe switching unit switches the output according to the interframe/intraframe coding distinc- 
55 tion flag 203. A predicted image 212 that is used for executing the interframe coding is synthesized by a predicted image 
synthesizer 211. Here, the decoded image 21 0 of the preceding frame stored in the frame memory 209 is subjected to 
a processing of moving the position of each of the blocks according to the motion vector 202 that is received. In the case 
of intraframe coding, on the other hand, the interframe/intraframe switching unit outputs the "0" signal 213. 

Block matching is a motion compensation system that is now most widely utilized. When the whole image is 
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expanding, contracting, or turning, however, the motion vectors of all of the blocks must be transmitted, causing a prob- 
lem of low coding efficiency. To solve this problem, global motion compensation (e.g.. M. Hotter, "Differential Estimation 
of the Global Motion Parameters Zoom and Pan". Signal Processing. VoL 16, No. 3. pp. 249-265. Mar.. 1989) has been 
proposed to express the motion vector field of the whole image while not using many parameters. According to this 
5 motion compensation system, the motion vector (ug(x. Y), vg(x, y)) of a pixel (x. y) in an image is expressed in the form 
of: 



u g (x, y) = a 0 x + atf + a 2 Equation 1 

10 v g (x.y) = a 3 x + a 4 y = a 5 

or 

u g ( x » y) = b 0 xy + b 1 x = b 2 y + b 3 Equation2 

15 

v g (x. y) =b 4 xy + b 5 x + b 6 y + b 7 

and the motion compensation is executed using the motion vectors. In these equations, aO to a5 and bO to b7 are 
motion parameters. In executing the motion compensation, the same predicted image must be generated both on the 

20 transmitting side and on the receiving side. For this purpose, the transmitting side may directly transmit values of aO to 
a5 or bO to b7 to the receiving side or may instead transmit motion vectors of several representative points. 

As shown in Fig. 3A, assume that the coordinates of the pixels at the left upper, right upper, left lower and right 
lower corners of an image 301 are expressed by (0, 0), (r, 0), (0, s) and (r. s) (where r and s are positive integers). Here, 
letting the horizontal and vertical components of the motion vectors of the representative points (0. 0), (r, 0) and (0. s) 

25 be (ua, va), (ub, vb) and (uc, vc). respectively. Equation 1 is rewritten as: 



u g (x,y) = r x+ c s a y+u a Equation 3 

30 

v g (x.y) = -° jr - a x+-S i - a y+v a 

This means that the same function can be fulfilled even when ua, va, ub. vb. uc and vc are transmitted instead of 
35 transmitting aO to a5. This state is shown in Figs. 3A and 3B. The motion vectors 306. 307 and 308 (the motion vectors 
are defined to start from points of the original image of the present frame and ends at the corresponding points in the 
reference image) of the representative points 303, 304 and 305 may be transmitted instead of the motion parameters 
based on the assumption that global motion compensation between the original image 302 of the present frame shown 
in Fig. 3B and the reference image 301 shown in Fig. 3A is effected. Similarly, by using the horizontal and vertical com- 
40 ponents (ua. va). (ub, vb), (uc. vc) and (ud. vd) of four representative points (0, 0), (r, 0), (0. s) and (r, s), Equation 2 can 
be rewritten as: 



u 9< x ' ^ - ?(T U a + ^b)+^T u c + 7 u d) Equation 4 



45 "a^b^c+Ud - u a +u b -U a +U c 

= — ii — — — x+ ^— y +u < 



v a (x. y) = 



so 



rs 



-xy+ — —^x+ — - — y+v a 



Therefore, a similar function is fulfilled even when ua. va, ub. vb. uc. vc. ud and vd are transmitted instead of bO to 
b7. In this specification, the system using Equation 1 is referred to as global motion compensation based upon linear 
interpolation and7or extrapolation, and the system using Equation 2 is referred to as global motion compensation based 
55 upon the bilinear interpolation and/or extrapolation. 

Fig. 4 illustrates the constitution of a motion compensation section 401 of an image coder employing the global 
motion compensation system based upon linear interpolation and/or extrapolation for transmitting motion vectors of the 
representative points. The same components as those of Fig. 1 are denoted by the same reference numerals. A video 
coder that executes global motion compensation can be constituted by substituting a motion compensation section 401 
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for the block matching unit 1 16 of Fig. 1 . 

A global motion compensation unit 402 performs motion estimation related to the global motion compensation 
between the decoded image 11 5 of the preceding frame and the original image 1 01 of the present frame, and estimates 
the values ua, va. ub, vb, uc and vc. The data 403 related to these values are transmitted as part of the motion data 

5 120. A predicted image 404 of global motion compensation is synthesized using Equation 3, and is fed to a block match- 
ing unit 405. The motion is compensated by block matching between the predicted image of global motion compensa- 
tion and the original image of the present frame, thereby generating motion vector data 406 of blocks and a final 
predicted image 1 1 7. The motion vector data and the motion parameter data are multiplexed through a multiplexing unit 
407 and are output as motion data 1 20. 

w Fig. 5 illustrates the constitution of a motion compensation section 50 1 which is different from that of Fig. 4. A video 
coder that executes global motion compensation can be constituted by substituting a motion compensation section 501 
for the block matching unit 1 16 of Fig. 1. In this embodiment, block matching is not adopted for the predicted image of 
global motion compensation but either global motion compensation or block matching is adopted for each of the blocks. 
Global motion compensation and block matching are executed in parallel by the global motion compensation unit 502 

is and the block matching unit 505 between the decoded image 1 1 5 of the preceding frame and the original image 101 of 
the present frame. A selection switch 508 selects an optimum system for each of the blocks between the predicted 
image 503 of global motion compensation and the predicted image 506 of block matching. The motion vectors 504 of 
the representative points, motion vectors 507 of the blocks and selection data 509 of global motion compensation/block 
matching are multiplexed by the multiplexing unit 510 and are output as motion data 120. 

20 By introducing the above-mentioned global motion compensation, it becomes possible to express the general 
motion of the image using few parameters and to accomplish a high data compression ratio. However, the amounts of 
coding processing and decoding processing become larger than those of the conventional systems. In particular, the 
division in Equations 3 and 4 is a major factor of complexity in the processing. 

25 Summary of the Invention 

The global motion compensation in which the motion vector field of the whole image is approximated by a few 
parameters, involves a problem of increasing the amount of processing involved in synthesizing a predicted image. The 
object of the present invention is to decrease the amount of the processing by substituting a binary number shift oper- 
30 ation for the division in the global motion compensation. 

The division is realized by the shift operation by suitably selecting the coordinates of representative points at the 
time of executing the global motion compensation. 

Brief Description of the Drawings 

35 

Fig. 1 is a diagram illustrating the constitution of an H.261 video coder. 
Fig. 2 is a diagram illustrating the constitution of an H.261 video decoder. 

Figs. 3A and 3B are diagrams illustrating an example of global motion compensation for transmitting the motion 
vectors of representative points. 
40 Fig. 4 is a diagram illustrating a motion compensation section of the video decoder for effecting the block matching 
of a predicted image of global motion compensation. 

Fig. 5 is a diagram illustrating a motion compensation section of the video coder for selecting either the global 
motion compensation or the block matching for each of the blocks. 

Fig. 6 is a diagram illustrating the arrangement of representative points for executing high-speed processing. 
45 Fig. 7 is a diagram illustrating a region for finding the motion vectors by extrapolation within the image. 

Fig. 8 is a diagram illustrating an example which the motion vectors of all pixels in the image are found by interpo- 
lation from the motion vectors of the representative points. 

Fig. 9 is a diagram in which the image is divided into two right triangles, and the global motion compensation is exe- 
cuted for each of them based on interpolation from the motion vectors or the representative points. 
so Fig. 10 shows a flow chart for performing video coding accorcfing to an embodiment of the invention. 

Fig. 1 1 shows a flow chart for performing video coding according to another embodiment of the invention. 
Fig. 12 shows a flow chart for video decoding according to an embodiment of the invention. 
Fig. 13 shows a flow chart for video decoding according to another embodiment of the invention. 
Fig. 14 is a diagram of a software encoder for a video coding method according to an embodiment of the invention. 
55 Fig. 15 is a diagram of a software decoder for a video decoding method according to the present invention. 
Fig. 16 is an overall diagram of a video encoder of the present invention. 

Fig. 17 is a diagram of a motion compensation unit used in the encoder or Fig. 16. according to one embodiment 
of the invention. 

Fig. 18 is a diagram of a motion compensation unit used in the encoder of the invention shown in Rg. 16 according 
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to another embodiment of the invention. 

Rg. 19 is a diagram of a video decoder for decoding a video signal according to the present invention. 

Fig. 20 is a diagram of a predicted image synthesizer used in the video decoder of Rg. 1 9 according to one embod- 
iment of the invention. 

5 Rg. 21 is a diagram of a precficted image synthesizer used in the video decoder of Fig. 19 according to another 

embodiment of the present invention. 

Detailed Description of the Preferred Embodiments 

io In the following description, the sampling intervals for the pixels are 1 in both the horizontal and vertical directions, 
and the coordinates of the pixels at the left upper, right upper, left lower and right lower corners are expressed by (0, 0), 
(r, 0), (0. s) and (r, s) (where r and s are positive integers), respectively, as shown in Fig. 3A. 

If the motion vector Is quantized for each of the pixels in compensating the motion based upon the linear interpola- 
tion and/or extrapolation (affine transform) or the bilinear interpolation and/or extrapolation (bilinear transform), mis- 

15 matching can be prevented and the operation can be simplified (Japanese Patent Application No. 193970/1994). In the 
following description, it is assumed that the horizontal component and vertical component of a motion vector of a pixel 
are integral multiples of 1/m (where m is a positive integer). Further, it is assumed that the global motion compensation 
is executed using the motion vectors of representative points explained in the "Prior Art" and that the motion vectors of 
the representative points are integral multiple of t/k (where k is a positive integer). In this specification, the "motion vec- 

20 tors of pixels" are the motion vectors that are actually used for synthesizing a predicted image. On the other hand, the 
"motion vectors of representative points" are the parameters used for calculating the motion vectors of pixels. Because 
of the difference in the quantization step sizes, therefore, the motion vectors of pixels often may not be in agreement 
with the motion vectors of representative points even though they have the same coordinates. 

With reference to Rg. 6, a case based upon linear interpolation and/or extrapolation will be explained. Here, as 

25 described in the "Prior Art", the representative points are not those located at the corners of the image 601 . but are the 
points 602. 603 and 604 having the coordinates (i. j), (i+p. j) and (i. j+q) (i. j, p and q are integers), respectively. At this 
moment, the points 602. 603 and 604 may exist inside or outside the image. Letting the coordinates whose values are 
given by multiplying the horizontal and vertical components of the motion vectors of the representative points by k be 
respectively (uO, vO), (u1, v1) and (u2, v2) (uO, vO, u1, vl, u2 and v2 are integers), coordinates (u(x. y) and v(x, y) 

30 (where x. y, u(x, y) and v(x, y) are integers) which are m times the horizontal and vertical components of the motion vec- 
tor of a pixel (x, y) are expressed by the following equations: 

u(x. y) = ((u1-u0)(x-i)q+(u2-u0)(y-j)p-KJ0pq)m//(pqk) Equation 5 

35 v(x, y) = ((v1-v0)(x-i)q+(v2-v0)(y-j)p+v0pq)m//(pqk) 

where 7f represents division for rounding the quotient of ordinary division into an adjacent integer when the quotient is 
not an integer, and its priority as an operator is the same as that of multiplication and division. To decrease the operation 
error, it is desirable that a value which is not an integer is rounded to the most adjacent integer. In this case, the methods 
40 for rounding a value of the sum of an integer and 1/2 are: 

(1) Rounding the value toward 0; 

(2) Rounding the value away from 0; 

(3) Rounding the value toward 0 when the dividend is negative, and rounding the value away from 0 when the div- 
45 idend is positive (assuming that the divisor is positive at all times); and 

(4) Rounding the value away from 0 when the dividend is negative, and rounding the value toward 0 when the div- 
idend is positive (assuming that the divisor is positive at all times). 

Among them, (3) and (4) are advantageous from the standpoint of processing quantity since the rounding direction 
so does not change irrespective of whether the dividend is positive or negative and there is no need to judge whether the 
sign is positive or negative. High-speed processing according to method (3) can be realized by the following equation: 

u(x, y) = (Lpqk+((u1 -u0)(x-i)q +(u2-u0)(y-j)pw-u0pq)m+(pqk#2))#(pqk)-L Equation 6 

55 v(x. y) = (Mpqk+((v1-v0)(x-i)q +(v2-v0)(y-j)p+v0pq)m+(pqk#2))#(pqk)-M 

where represents division of an integer for rounding off the decimal part toward 0, which is. usually, most easily real- 
ized using a computer. L and M are sufficiently large positive integers for maintaining the dividend of division to be pos- 
itive at all times. The term (pqk # 2) is used for rounding the quotient of division to the most adjacent integer. 
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Integer processing contributes to decreasing the amount erf processing. Here, assuming that p. q and k are 2 a , 2 P 
and 2*°. respectively wherein a and p are positive integers, and hO is an integer which is not negative. The division of 
Equation 5 can be realized by the shift operation of a+p+hO bits, making it possible to greatly decrease the amount of 
processing using a computer or dedicated hardware. Furthermore, assuming that m is 2 h1 (hi is an integer which is not 
5 negative, and hi < a+p+hO ). Equation 6 can be rewritten as: 

u(x. y) = ((2L+1)«(a + p + hO-hM)+(u1-uO)(x-i) «p Equation 7 

+(u2-u0)(y-j)«a+u0«(a+p)) >>(a+p+hO-h1)-L 

10 v(x. y) = ((2M+1)«(a+p+hf>h1.1>+<v1-vO)(xH) «p 

+(v2-v0)(y-j)<<a+v0<<(a+p)) »(a+p+hO-h1)-M 



where "x « a" means that x is shifted left by a bits and 0 is substituted for the low-order a bits, "x » a" means that x 
is is shifted right by a bits and 0 or 1 is substituted for the high-order a bits (when x is a number of complement represen- 
tation of 2, 1 is substituted when the most significant bit of x is 1 and 0 is substituted when it is 0). and the priority of 
these operators lies between addition/subtraction and multiplication/division, making it possible to further simplify the 
operation. 

When the linear interpolation and/or extrapolation is used, letting (u3. v3) be the coordinates determined by murti- 
20 plying the horizontal and vertical components of a motion vector of a representative point at (i+p, j+q) by k. Equation 5 
is rewritten as Equation 8 or Equation 9, as follows: 



y) = -u0)(x-i)q^u3-u1){y-j>+u0rxi)rn//(pqk) Equation 8 

25 v(x. y) = ((v1 -v0)(x-i)q+(v3-v1 )(y-j)p+vOpq)m//(pqk) 

where the representative points are: 
(i. j). C»+P. 0 and (i+p. j+q). 

30 u(x, y) = ((u3-u2)(x-i)q+(u2-u0)(y-j^u0pq)rn//(pqk) Equation 9 

v(x, y) = ((v3-v2)(x-i)q+(v2-v0)(y-j)p+v0pq)m//(pqk) 

where the representative points are: 
35 (i. D. (i. i+q) and (i+p. j+q). 
or 



u(x. y) = ((u2-u3)(i+p-x)q+(u1-u3)Q+q-y)p+u3pq)m//(pqk) Equation 10 

40 v(x, y) = ((v2-v3)0+p-x)q+(v1-v3)Q+q-y)p+v3pq)m//(pqk) 

where the representative points are: (i+p. j), (i, j+q) and (i+p, j+q). 

making it possible to decrease the amount of processing by using p, q, k and m which are numbers of 2 n (where n is a 
positive integer). 

45 When the bilinear interpolation and/or extrapolation are used, letting (uO. vO). (u1 , v1), (u2, v2) and (u3. v3) be the 

coordinates determined by multiplying the horizontal and vertical components of the motion vectors of the representa- 
tive points (i, j), (i+p. j), (i, j+q) and (i+p. j+q) by k, u(x, y) and v(x. y) are represented by the following equation: 

u(x. y) = (G+q-y)(('+p-x)u0+(x-i)u1) +(y-j)((i+p-x)u2+(x-i)u3))m//(pqk) Equation 1 1 

so 

v(x. y) = (0+q-y)((i+p x)vO+(x-i)v1) +(y-j)((i+p-x)v2+(x-i)v3))m//(pqk) 
Equation 1 1 can be rewritten as: 



55 u(x. y) = ((2L+1)«(a+p+hO-h1-1)+Q + q-y)((i+p-x)uO Equation 12 

+(x-i)u1)+(y-j)((i+p-x)u2+(x-i)u3)) »(a+p+h0-h1)-L 
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v(x, y) = ((2M+1)«(a+p+hO-h1-1)+Q+q-y)((i+p-x)vO 
+<x-i)v1 )-Ky-fl(G+p-x)v2+<x-j)v3)) »(a+p+hO-h1)-M 



5 by using p, q, k and m which are numbers of 2 a . 2 P . 2 h0 , and 2 h1 , respectively, making it possible to decrease the 
amount of processing as in the above-mentioned processing. 

In order to obtain the same predicted image of global motion compensation on the transmitting side and on the 
receiving side, the data related to the motion vectors of the representative points must be transmitted in a certain form 
to the receiving side. The motion vectors of the representative points may be directly transmitted. It is, however, also 

10 possible to transmit the motion vectors of the corner points of the image and to calculate the motion vectors of the rep- 
resentative points therefrom. This method will now be described. 

First a case where the linear interpolation and/or extrapolation is employed will be described. It is assumed that the 
motion vectors of three corner points (0. 0), (r. 0) and (0, s) of the image take only those values which are integral mul- 
tiples of 1/n, and that the coordinates (uOO. vOO), (u01. v01) and (u02. v02) which are determined by multiplying the hor- 

is izontal and vertical components by n are transmitted. In this case, the coordinates (uO. vO), (u1 , v1). (u2. v2) and (u3, 
v3) which are determined by multiplying the horizontal and vertical components of the motion vectors by k are defined 
as follows: 

uO = u'(i, j) Equation 1 3 

20 

vO = v*(i. 0 
u1 = u'(i+p. j) 

25 V1 = V f (i+P. fl 

u2 = u'fl.j+q) 
v2 = v (i, j+q) 

30 

u3 = u'(i+p. j+q) 
v3 = v*(i+p, j+q) 

35 where u'(x, y) and v*(x, y) are defined by the following equation, which is a modification of Equation 5: 

u'fr y) = ((u0 1 -u00)xs+(u02-u00)yr+u00rs)k///(rsn) Equation 1 4 

V(x. y) = ((V01-v00)xs+(v02-v00)yr+v00rs)k///(rsn) 

40 

Here, 7/f represents division for rounding the quotient of an ordinary division into an adjacent integer when the quotient 
is not an integer, and its priority as an operator is the same as that of multiplication and division. Three points are 
selected out of (uO, vO). (u1. v1), (u2. v2) and (u3. v3). and the global motion compensation is executed using such 
points as representative points. Then, the global motion compensation can be approximated by using (0, 0), (r, 0) and 

45 (0, s) as the representative points. Here, by using p and q which are 2 n (n is positive integer), the processing can be 
simplified as described earlier. In order to decrease the operation errors, it is desirable that 7/T rounds a value which is 
not an integer into the most adjacent integer. In this case, methods for rounding a value of the sum of an integer and 
1/2 include the above-mentioned methods (1) to (4). Compared to the case using Equation 5 (calculation for each pixel), 
however, the operation of Equation 14 (only three calculations for one image) does not require many calculations. Even 

so if methods (1) or (2) are selected, therefore, the total amount of calculation is not greatly affected. 

When three points different from those of the case using Equation 1 3 are selected as corner points of the image, 
the same processing can be realized by modifying Equations 8 to 10. In addition to the above-mentioned examples, by 
letting (u03. v03) be the coordinates determined by multiplying the horizontal and vertical components of a motion vec- 
tor at a corner point (r. s) of the image by n. Equation 14 can be rewritten as: 

55 

u'(x. y) = ((u01 -u00)xs+(u03-u01 )yr+u00rs)k///(rsn) Equation 1 5 

V(x, y) = ((v01-v00)xs+(v03-v01)yr+v00rs)k///(rsn) 
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when (uOO. vOO). (u01 , v01) and (u03, v03) are transmitted; can be rewritten as: 

y) = ((u03-u02)xs+(u02-u00)yr-KJ00rs)k///(rsn) Equation 16 

5 v"(x f y) = ((V03-v02)xs+(v02-v00)yr+v00rs)k///(rsn) 

when (uOO, vOO), (u02, v02) and (u03, v03) are transmitted; and can be rewritten as: 

u f (x. y) = ((u02-u03)(r-x)s+(u01 Hj03)(s-y)r+u03rs)k ///(rsn) Equation 1 7 

10 

V(x. y) = ((V02-v03)(r-x)s+(v01-v03)(s-y)r+v03rs)k///(rsn) 

when (u01, v01), (u02. v02) and (u03, v03) are transmitted. 

The same also holds even when the bilinear interpolation and/or extrapolation are executed. As in the above- men - 

is tioned case, assume that the motion vectors of the four corner representative points (0, 0), (r. 0), (0, s) and (r. s) of the 
image take only those values which are integral multiples of 1/n, and that (uOO, vOO). (u01. v01), (u02, v02) and (u03, 
v03) which are n times the horizontal and vertical components of the representative points are transmitted. In this case, 
(uO, vO), (u1 . v1), (u2, v2) and (u3, v3) which are k times the horizontal and vertical components of the motion vectors 
of the representative points (i, j), (i+p, j), (i, j+q) and (i+p. j+q) are given by Equation 13 as described above. Here, how- 

20 ever, by modifying Equation 11, 
u*(x. y) and v'(x, y) are defined by: 

u'(x. y) = ((s-y)((r-x)u00+xu0 1 )+y((r-x)u02+xu03))k ///(rsn) Equation 1 8 

25 v'(x. y) = ((s-y)((r-x)v00+xv01 )+y((r-x)v02+xvO3))k ///(rsn) 

The advantage of the method in which the motion vectors of corner points of the image are transmitted and are 
interpolated and/or extrapolated to f ind motion vectors of representative points, is that the ranges of the motion vector 
for each of the pixels can be easily limited. For example, in the case of the bflinear interpolation and/or extrapolation 

30 given by Equation 4, the value ug(x, y) is not greater than the maximum value of ua. ub, uc and ud and not smaller than 
the minimum value thereof when the point (x. y) is inside the image. Therefore, if a limiting condtion is added so that 
the values ua. ub, uc and ud lie within a limited range (e.g., range within ±32 pixels) at the time of estimating the global 
motion, the value ug(x. y) can be confined within the same range for all pixels (this also holds even for vg(x, y), as a 
matter of course). This makes it possible to definitely determine the number of digits necessary for the calculation. 

35 which is convenient from the standpoint of designing software or hardware. 

The foregoing description, however, is based upon the case that the calculations are all carried out based upon 
using floating-point arithmetic operations and, hence, care must be given in practice. The arithmetic operation (Equa- 
tion 18) for finding the motion vectors of representative points from the motion vectors of corner points of the image 
involves rounding a value into an integer. Therefore, consideration must be taken to the probability that the motion vec- 

40 tors found by Equation 1 2 may deviate out of the above-mentioned limited range due to the calculation error. In partic- 
ular, care must be taken when the representative points are located inside the image. This is because, the motion 
vectors are found by the extrapolation for the pixels outside a rectangle defined by representative points and, hence, the 
rounding error may be amplified. 

Fig. 7 illustrates an example in which the motion vectors are found by extrapolation. When the global motion com- 

45 pensation is executed for the image 701 by using representative points 702, 703, 704 and 705, the motion vectors are 
calculated by the extrapolation for the hatched portions inside the image. This is because, the hatched portions exist 
outside a rectangle 706 defined by the representative points. 

This problem can be effectively solved by so arranging the four representative points that a rectangle defined by the 
representative points includes the whole image. This is shown in Fig. 8. A rectangle 806 defined by representative 

so points 802, 803, 804 and 805 includes an image 801 . Then the motion vectors of all pixels can be found by the interpo- 
lation from the representative points, and the effect of the rounding error at the representative points is not amplified 
inside the Image. Accordingly, an error larger than the rounding error at representative points never occurs inside the 
image, and the upper limit of error is definite. When the rectangle defined by the representative points is too large, how- 
ever, the range of values that the motion vectors of representative points take is so wide that a number of digits neces- 

55 sary for the arithmetic operation increases, causing a disadvantage from the standpoint of mounting. 

From the foregoing description, it is desirable that the value p is larger than r and the value q is larger than s in order 
to decrease the effect of the rounding error. It is also desirable that p and q assume values that are as large as possible 
even when they are smaller than r and s. It is further desirable that the values i and j are such that a portion which is as 
wide as posstole inside the image is in an area that is defined by the representative points. 
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When the bilinear interpolation and/or extrapolation are used for the global motion compensation as described 
above, the components of motion vectors of pixels in the rectangle defined by the two representative points can take 
only values that lie between maximum values and minimum values of the components of the motion vectors of the rep- 
resentative points. When linear interpolation and/or extrapolation is used, on the other hand, the motion vectors of pix- 
5 els in a triangle defined by three representative points have the game property. When the global motion compensation 
is executed by using the linear interpolation and/or extrapolation, therefore, it is effective to transmit the motion vectors 
of the four corner points of the image and to carry out the global motion compensation independently for the two right 
triangles divided by a diagonal of the image. Then, the limitation on the range of the motion vectors of the four corner 
points can be directly applied to the motion vectors of ail pixels inside the image. In this case, the values i. j, p and q 
io may not be the same between the two right triangles. From the standpoint of operation error, furthermore, it is desirable 
that triangles defined by the representative points include right triangles of which the global motion compensation is to 
be executed, respectively, in order to avoid the calculation of motion vectors of pixels by the extrapolation. This is shown 
in Fig. 9. The motion vectors of points 909, 903, 908 and 910 which are the four corners of an image 901 are transmit- 
ted, and the global motion compensation is independently executed for each of a right triangle defined by the points 
, 5 909, 903 and 910 and a right triangle defined by the points 909, 910 and 908. Therefore, if a limitation is imposed on 
the range of motion vectors of vert exes, the motion vectors of all pixels within the image are included in this limited 
range. The right triangle defined by the points 909, 903 and 91 0 uses points 902, 903 and 904 as representative points, 
and the right triangle defined by the points 909, 910 and 908 uses points 906. 907 and 908 as representative points. 
The triangles defined by the representative points include therein right triangles to which the global motion compensa- 
te tion is to be executed, respectively Therefore, the effect of the rounding error of the motion vectors of representative 
points is not amplified at points inside the image. In this example, the two triangles defined by the representative points 
are similar to each other. However, the triangles may not necessarily be similar to each other. 

The present invention makes H possible to substitute the shift operation for the division for synthesizing a predicted 
image of global motion compensation, and to simplify the processing using either software or dedicated hardware or a 
25 combination of both. 

Fig. 10 shows the steps followed in performing video coding of video image data using fast global motion compen- 
sation according tO an embodiment of the present invention. In step 150. a video signal is input and in step 151. global 
motion estimation is performed between an input image and the decoded image of a previous frame. Then, the motion 
vectors are derived from the representative points of the input image in step 1 52. 
30 In the next step, step 1 53, a predicted image of global motion compensation is synthesized using the fast algorithm. 
The fast algorithm is a general expression for algorithms disclosed herein, such as the bilinear algorithm and affine 
algorithm. For example, equation 1 is an affine algorithm whereas equation 2 is a bilinear algorithm. Further, equations 
3. 5. 6, 7-10, and 14-17 are affine whereas equations 4. 1 1 and 18 are bilinear. 

In step 154, the local motion estimation is performed between the input image and the decoded image of the pre- 
35 vious frame. The predicted image of local motion compensation is synthesized in step 155 and the global or local 
motion compensation for each block is selected in step 156. The selection step is necessary since the global motion 
compensation and local motion compensation steps are performed in parallel in this embodiment 

Then, in step 157, the error image is synthesized by calculating the difference between the predicted image and 
the input image and the error image is subject to a discrete cosine transform for quantizing the DCT coefficients in step 
40 1 58. Finally, in step 1 59, the compressed video data is output. 

In Fig. 1 1 , an alternative embodiment is disclosed for performing video coding, which is similar to the video coding 
disclosed in Fig. 10. Specifically, steps 150-153 are the same, but the remainder of the steps shown in the flow chart 
are different. The reason for this is that the steps performed in Fig. 1 1 are for performing the local motion compensation 
and global motion compensation serially, rather than in parallel as in Fig. 10. Accordingly, in step 254, local motion esti- 
45 mation is performed between the input image and the predicted image of global motion compensation. Then, in step 
255, the predicted image of local motion compensation is synthesized. Finally, the error image is synthesized by calcu- 
lating the difference between the predicted image and the input image, just as in step 157 in Fig. 10. and steps 257 and 
258 are the same as steps 158 and 159, explained above. 

Fig. 12 shows a flow chart of the video decoding according to the present invention. In step 160, an input bit stream, 
so such as a h.261 bit stream is received as the compressed video data. The motion vectors of the representative points 
are derived and in step 161 and in step 162, the predicted image for blocks which selected global motion compensation 
using the fast algorithm are selected. In step 164. the predicted image for blocks which selected local motion compen- 
sation are synthesized. The error image with respect to the predicted image is synthesized in step 165 and the error 
image is added to the predicted image in 166. In step 167, the reconstructed video signal is output to complete the 
55 decoding of the encoded video data. 

According to the embodiment of Fig. 12, the synthesizing of the predicted image for blocks which selected global 
motion compensation using the fast algorithm and also for blocks which selected local motion compensation is per- 
formed in parallel. On the other hand, in Fig. 13, the flow chart shows an alternative embodiment in which these steps 
are performed serially. 
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In Fig. 13, steps 160 and 161 are the same as those in Fig. 12. In step 262. the predicted image of global motion 
compensation using the fast algorithm is synthesized and in step 263 the predicted image of local motion compensation 
is synthesized. These steps are performed serially and followed by the step of synthesizing the error image by applying 
inverse DCT to the DCT coefficients, which is the same as step 165 in Fig. 1 2. Steps 265 and 266 which follow are also 
5 the same as steps 166 and 167 discussed with respect to Fig. 12, and which result in the output of the reconstructed 
video signal. 

Figs. 1 4 and 1 5 are block diagrams of the components of the encoder and decoder of the invention for storing and 
executing software operating as disclosed in the flowcharts of Figs. 10-13. The components in common for both da- 
grams have the same reference numbers and include the data bus 1 40. CPU 1 42 and storage device 143. The encoder 

10 program for executing the video encoding is shown in Fig. 1 4 t and is stored in storage device 143. The decoder program 
for executing the video decoding is shown in Fig. 15. and is stored in storage device 143. Storage devices 143 are stor- 
age media, such as hard disk drives, floppy disks or optical disks, for example. 

With reference to Fig. 14. an input video signal is A/D converted by A/D converter 141 and sent to CPU 142 over 
bus 140. CPU 142 retrieves and executes the encoder program 144 stored in storage device 143 and then encodes and 

is compresses the video data received from the A/D converter 141 . After the video data is encoded, it is stored in an out- 
put buffer 145 and output as output data. Control data and timing signals are also output with the compressed video 
data. 

Fig. 15 shows the processing of coded video signal, which is received at input buffer 148 and then read by CPU 
142. CPU 142. which retrieves the decoder program 147 from the storage device 143. executes the decoding of the 
20 coded video data. The decoded video data is then sent over bus 140 to D/A converter 146 for outputting an analog 
video signal. 

Fig. 16 shows the overall block diagram of a video coder according to the invention that is similar to Fig. 1 of the 
prior art. Accordingly, the components in common for both diagrams have the same reference numbers. In the diagram 
of Fig. 16, block 1 16 of Fig. 1 . which is a block matching unit for local motion compensation, is replaced with a block 

25 1002 that uses global motion compensation and local motion compensation. Otherwise, the remaining components in 
the Fig. 1 6 diagram are the same as those in the cSagram of Fig. 1 . 

In Fig. 17. a motion estimation and compensation unit 1003 that performs serial processing is shown. Unit 1003 
can be used as the motion estimation and compensation unit 1 002 of Fig. 16. Further, unit 1003 is a hardware embod- 
iment performing functions nearly equivalent to the steps performed in the software processing shown in Fig. 1 1. 

30 As shown in Fig. 1 7. an input video signal 1 0 1 is received by the global motion estimation unit 1 004 and also by the 
block matching unit 405. Global motion estimation is performed between an input image and the decoded image of a 
previous frame by the global motion estimation unit 1004. Unit 1004 also derives the motion vectors from the represent- 
ative points of the input image. The data 403 related to these values is transmitted to the global motion compensation 
(GMC) image synthesizer 1005 which synthesizes the predicted image of the global motion compensation using the 

35 fast algorithm. A predicted image 404 of global motion compensation is then output to block matching unit 405 in which 
local motion estimation between the input image and the predicted image of global motion compensation is performed. 
Then, the motion vector data 406 is output to the multiplexer 407 and the predicted images of the present frame 1 1 7 is 
output to the adder 102 for synthesizing the error image by calculating the difference between the predicted image and 
the input image. The motion compensation unit shown in Fig. 17 uses serial global motion estimation and local motion 

40 estimation. 

In Fig. 18, a motion compensation unit 1006 which can be used as the motion compensation unit 1002 in Fig. 16 
is disclosed in which parallel processing is performed for the global motion estimation unit and the local motion estima- 
tion, as follows. First, a video signal 101 is input and received by both global motion estimation unit 1008 and block 
matching unit 505. Then, global motion estimation is performed between the input image and the decoded image of the 

45 previous frame by the global motion estimation unit 1008. The motion parameters 504, such as the motion vectors of 
representative points are input to the multiplexer 510 and the global motion compensation (GMC) image synthesizer 
1007. A predicted image of global motion compensation using the fact algorithm is synthesized and output to a block 
matching/global motion compensation changeover switch 508 for outputting the predicted image of the present frame 
117. obtained by one of global or local motion compensation for each block. The selection data 509 of the changeover 

so switch selection is output to the multiplexer 510. The multiplexer also receives the output of 507 of block matching unit 
505. which is the motion vector data. A signal 120 is output from the mutiplexer that includes signals 504. 507 and 509. 

Fig. 1 9 shows a block diagram of a video decoder that is similar to the prior art decoder of Fig. 2. but that includes 
the addition of a predicted image synthesizer 1010 which synthesizes the predicted image in accordance with an 
embodiment of the present invention. Otherwise, the remaining components of the decoder 1009 are the same as 

55 shown in Fig. 2. 

In Fig. 20. a predicted image synthesizer according to one embodiment of the invention 101 1 is shown, which can 
be used for the predicted image synthesizer 1010 shown in Fig. 19. Serial processing is shown in Fig. 20. in which the 
motion vector data 202 is received by the multiplexer 1013. which provides the motion parameters 403 and motion vec- 
tor data 406 to the global motion compensation (GMC) image synthesizer 1005 and the block matching image synthe- 
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sizer 1012. respectively. The GMC image synthesizer 1005 derives the motion vectors of representatives points and 
synthesizes a predicted image of global motion compensation using the fast algorithm. Then, it outputs the predicted 
image of global motion compensation 404 to the BM image synthesizer 1012, which synthesizes the predicted image 
of local motion compensation. The predicted image of the present frame 212 is then output to the switching unit 214, 
5 as shown in Fig. 19. 

Fig. 21 shows a predicted image synthesizer 1014, which operates to process the global motion compensation 
image synthesizing and block matching image synthesizing in parallel, as follows. 

The motion vector data 202 is input to the multiplexer 1016. which provides separated motion parameter data 504, 
motion vector data 507 and selection data of block matching/global motion compensation 509 to the GMC image syn- 
io thesizer 1007. BM image synthesizer 1015 and switch 508, respectively, as shown. The BM image synthesizer 1015 
synthesizes the predicted image for blocks which selected the local motion compensation and the GMC image synthe- 
sizer 1007 synthesizes the predicted image for blocks which selected the global motion compensation using the fast 
algorithm. The respective data 503 and 506 is output to the switch 508, which selects one of these signals according to 
the selection data 509, received from the demultiplexer. The predicted image of a present frame 21 2 is then output and 
is received by switching unit 214, as shown in Fig. 19. 

According to the embodiments of the invention, the video coding and decoding can be performed either by software 
operating as shown in the flowcharts of Figs. 10-13 using the software encoder or software decoder shown in Figs. 14 
and 15 or by dedicated hardware, as shown in the embodiments of the invention in Figs. 16-21. 

20 Claims 

1 . A method of global motion compensation, comprising: 

calculating the motion vectors for all pixels in an image by effecting linear interpolation and/or extrapolation of 
25 the motion vectors at three or four representative points for the image, and 

assuming sampling intervals of the pixels to be 1 in both horizontal and vertical directions with respect to the 
image, and assuming sampling points for which horizontal and vertical components of coordinates of the sam- 
pling points are both integers, 

wherein the representative points are three or four of the points having the coordinates (i. 0. (i+P. j), (i. j+q). and 
30 (i+p, j+q). (where i and j are integers, and p and q are positive integers), and wherein p and q are 2 a and 2 P (a 

and p are positive integers), respectively. 

2. A method of synthesizing an interframe predicted image, comprising: 

35 sampling an image with sampling intervals of pixels in an image of 1 in both the horizontal and vertical direc- 

tions and with sampling points having horizontal and vertical components of the coordinates that are both inte- 
gers, 

wherein the horizontal and vertical components of motion vectors at representative points (i, j), (i+p, j) and (i. 
j+q) assume values of integral multiples of 1/k only (where i and j are integers, p and q are positive integers, k 
40 is 2*° and hO is an integer which is not negative), 

when the horizontal and vertical components of the motion vectors of the pixels assume values of integral mul- 
tiples of 1/m only (where m is 2 h1 , and hi is an integer which is not negative), 

using (uO, vO), (u1 , v1 ) and (u2, v2) (where (uO, vO, u1 , v1 , u2 and v2 are integers) which are k times the hor- 
izontal and vertical components of motion vectors at the representative points (i, j), (i+p. j) and (i, j+q). 
45 wherein (u(x, y) and v(x, y)) which are m times the horizontal and vertical components of the motion vector of 

a pixel (x, y) (where x. y. u(x, y) and v(x, y) are integers) are given by 

u(x. y) = (uO - p • q+(u1 -u0)(x-i) • q+(u2-u0)(y-j) - p)m//(p • q - k) 

50 v(x. y) = (vO • p • q+(v1 -vO)(x-i) • q+(v2-v0)(y-j) • p)m//(p • q • k) 

(where m ir represents a division for rounding the quotient of the ordinary division into an adjacent integer when 
the quotient of the ordinary division is not an integer, and its priority as an operator is the same as that of the 
ordinary multiplication and division) and 
55 wherein p and q are 2 a and 2 P (where a and f> are positive integers), respectively 

3. A method of synthesizing an interframe predicted image, comprising: 

sampling an image with sampling intervals of pixels in an image of 1 in both the horizontal and vertical direc- 
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tions and with sampling points having horizontal and vertical components of the coordinates that are both inte- 
gers. 

wherein the horizontal and vertical components of motion vectors at representative points (i, j). (i+p, j) and (i+p. 
j+q) assume values of integral multiples o1 1/k only (where i and j are integers, p and q are positive integers, k 
is 2 h0 , and hO is an integer which is not negative), 

when the horizontal and vertical components of the motion vectors of the pixels assume values of integral mul- 
tiples of 1/m only (where m is 2 h1 . and hi is an integer which is not negative). 

using (uO, vO), (ul. v1) and (u3 f v3) (where uO, vO, u1. v1, u3 and v3 are integers) which are k times the hori- 
zontal and vertical components of the motion vectors at the representative points (i, j). (i+p, j) and (i+p, j+q). 
wherein (u(x. y) and v(x, y)) which are m times the horizontal and vertical components of the motion vector or 
a pixel (x, y) (where x. y. u(x, y) and v(x, y) are integers) are given by 

u(x, y) = (uO • p • q+(u1-u0)(x i) • q+(u3-u1)(y-j) • p)m//(p • q • k) 

v(x. y) = (vO • p • q+{v1 -vO)(x-i) • q+(v3-v1)(y-j) • p)m//(p • q • k) 

(where 7T represents a division for rounding the quotient of the ordinary division into an adjacent integer when 
the quotient of the ordinary division is not an integer, and its priority as an operator is the same as that of the 
ordinary multiplication and division) and 

wherein p and q are 2 a and 2 P (where a and p are positive integers), respectively. 

A method of synthesizing an interframe predicted image, comprising: 

sampling an image with sampling intervals of pixels in an image of 1 in both the horizontal and vertical direc- 
tions and with sampling points having horizontal and vertical components of the coordinates that are both inte- 
gers. 

wherein the horizontal and vertical components of motion vectors at representative points (i. j). (i. j+q) and (i+p. 
j+q) assume values of integral multiples of 1/k only (where i and j are integers, p and q are positive integers, k 
is 2 h0 and ho is an integer which is not negative). 

when the horizontal and vertical components of the motion vectors of the pixels assume values of integral mul- 
tiples of 1/m only (where m is 2 h1 , and hi is an integer which is not negative), 

using (uO, vO), (u2, v2) and (u3. v3) (where uO. vO, u2. v2. u3 and v3 are integers) which are k times the hori- 
zontal and vertical components of the motion vectors at the representative points (i. j), (i. j+q) and (i+p, j+q), 
wherein (u(x, y) and v(x, y)) which are m times the horizontal and vertical components of the motion vector of 
a pixel (x, y) (where x, y. u(x. y) and v(x, y) are integers) are given by 

u(x. y) = (uO • p • q+(u3-u2)(x-i) • q+(u2-u0)(y-j) • p)m//(p • q • k) 

v(x, y) = (vO • p • q+<v3-v2)(x-i) • q+(v2-v0)(y-j) • p)m//(p • q • k) 

(where 7T represents a division for rounding the quotient of the ordinary division into an adjacent integer when 
the quotient of the ordinary division is not an integer, and its priority as an operator is the same as that of the 
ordinary multiplication and division) and 

wherein p and q are 2 a and 2 P (where a and p are positive integers), respectively. 

A method of synthesizing an interframe predicted image, comprising: 

sampling an image with sampling intervals of pixels in an image of 1 in both the horizontal and vertical direc- 
tions and with sampling points having horizontal and vertical components of the coordinates that are both inte- 
gers, 

wherein the horizontal and vertical components of motion vectors at representative points (i+p. j), (i, j+q) and 
(i+p. j+q) assume values of integral values of 1/k only (where i and j are integers, p and q are positive integers, 
k is 2 h0 . and hO is an integer which is not negative). 

when the horizontal and vertical components of the motion vectors of the pixels assume values of integral mul- 
tiples of 1/m only (where m is 2 h1 . and hi is an integer which is not negative). 

using (u1. v1). (u2, v2) and (u3, v3) (where u1, v1, u2. v2. u3 and v3 are integers) which are k times the hori- 
zontal and vertical components of the motion vectors at representative points (i+p, j), (i. j+q) and (i+p. j+q), 
wherein (u(x, y) and v(x, y)) which are m times the horizontal and vertical components of the motion vector of 
a pixel (x. y) (where x. y. u(x. y) and v(x. y) are integers) are given by. 
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u(x, y) = (u3 • p • q+(u2-u3)(p-x+i) • q+(u1-u3)(q-y+j) • p)m//(p • q • k) 
v(x, y) = (vO • p • q+(v2-v3)(p-x+i) • q+(v1-v3)(q-y+j) • p)m//(p • q • k) 

5 (where 7T represents a division for rounding the quotient of the ordinary division into an adjacent integer when 

the quotient of the ordinary division is not an integer, and its priority as an operator is the same as that of the 
ordinary multiplication and division) and 

wherein p and q are 2 a and 2 P (where a and p are positive integers), respectively. 

10 6. A method of synthesizing an interframe predicted image, conprising: 

sampling an image with sampling intervals of pixels in an image of 1 in both the horizontal and vertical direc- 
tions and with sampling points having horizontal and vertical components of the coordinates that are both inte- 
gers, 

is wherein the horizontal and vertical components of motion vectors at representative points (i, j), (i+p, j), (i, j+q) 

and (i+p, j+q) assume values of integral values of 1/k only (where i and j are integers, p and q are positive inte- 
gers, k is 2*°, and hO is an integer which is not negative), 

when the horizontal and vertical components of the motion vectors of the pixels assume values of integral mul- 
tiples of 1/m only (where m is 2 h1 , and hi is an integer which is not negative), 
20 using (uO, vO). (u1 , v1), (u2, v2) and (u3. v3) (where uO, vO. ul , v1 , u2, v2, u3 and v3 are integers) which are 

k times as great as the horizontal and vertical components of the motion vectors at representative points (i. j), 
(i+p. ft. p. j+q) and (i+p. j+q). 

wherein (u(x. y) and v(x, y)) which are m times the horizontal and vertical components of the motion vector of 
a pixel (x, y) (where x, y, u(x, y) and v(x, y) are integers) are given by. 

25 

u(x. y) = ((uO(p-x+0+u1(xH))(q-y+j)+ (u2(p-x+0+u3(x-i))(y-j*))m//(p * q • k) 

v(x. y) = ((v0(p-x+i)+v1(x-i))(q-y+j)+ (v2(p-x+i)+v3(x-i))(y-j))m//(p • q • k) 

30 (where 7T represents a division for rounding the quotient of the ordinary division into an adjacent integer when 

the quotient of the ordinary division is not an integer, and its priority as an operator is the same as that of the 
ordinary multiplication and division) and 

wherein p and q are 2 a and 2p (where a and p are positive integers), respectively. 

35 7. A method of synthesizing an interframe predicted image according to any of daims 2 to 5, wherein 

the coordinates of pixels at the left upper end, right upper end. left lower end and right lower end of the image 
are (0. 0). (r, 0), (0, s) and (r. s) (where r and s are positive integers). 

the horizontal and vertical components of motion vectors at corner points (0. 0), (r, 0) and (0, s) of the image 
40 take values of integral multiples of 1/n (where n is a positive integer). 

said method uses (uOO, vOO), (uOl . v01) and (u02, v02) (where u00, vOO. u01 , v01 , u02 and v02 are integers) 
which are n times said motion vectors, and 

(uO, vO), (u1 , v1), (u2, v2) and (u3. v3) which are k times the horizontal and vertical <x>mponents of motion vec- 
tors at points (i. j). (i+p, j). (i, j+q) and (i+p. j+q) and are given by, 

45 

u'(x, y) = (uOO • r • s+(u01-u00)x • s+(u02-u00)y • r)k ///(r • s * n) 
v'(x, y) - (vOO • r • s+(v01-v00)x • s+(v02-v00)y - r)k ///(r - s • n) 
so uO = u*(i, j) 

vO = v'(i. j) 
u1 = u'(i+p. j) 

55 

V1 = v'(i+p, j) 
u2 = u*(i. j+q) 
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v2 = V(i. j+q) 
u3 = u(i+p. j+q) 

5 v3 = v*(i+p. j+q) 

(where 7/r represents a division for rounding the quotient of the ordinary division into an adjacent integer when 
the quotient of the operation is not an integer, and its priority as an operator is the same as that of the ordinary 
multiplication and division) and 
io further wherein three points among these four points and their motion vectors are used as representative 

points and their motion vectors, respectively. 

8. A method of synthesizing an interframe predicted image according to any of claims 2 to 5. wherein 

is the coordinates of pixels at the left upper end. right upper end. left lower end and right lower end of the image 

are (0. 0). (r. 0). (0. s) and (r. s) (where r and s are positive integers). 

the horizontal and vertical components of motion vectors at comer points (0. 0). (r. 0) and (r, s) of the image 
take values of integral multiples of 1/n (where n is a positive integer), 

said method uses (uOO. vOO). (uOl. vOI) and (u03. v03) (where u00. vOO, u01 , v01, u03 and v03 are integers) 
20 which are n times as great as said motion vectors, and 

(uO. vO). (u1 . v1), (u2. v2) and (u3. v3) which are k times as great as the horizontal and vertical components of 
motion vectors at points (i. j), (i+P. fl. (». j+q) and (i+p, j+q) are given by. 

u*(x. y) = (uOO • r • s+(u01-u00)x • s+(u03-u01)y • r)k ///(r • s • n) 

25 

v'(x. y) = (vOO • r • s+(v01-v00)x • s+(v03-v01)y • r)k ///(r • s • n) 
uO = uTi. D 

30 vO = v'(i. j) 

u1 = uTnp. j) 
v1 = V(i+p. j) 

35 

u2 = u'(i. j+q) 
v2 = V(i. j+q) 

40 u3 = u*(i+p. j+q) 

v3 = V(i+p, j+q) 

(where 7/r represents a division for rounding the quotient of the ordinary division into an adjacent integer when 
45 the quotient of the operation is not an integer, and its priority as an operator is the same as that of the ordinary 

mutti plication and division) and 

further wherein three points among these four points and their motion vectors are used as representative 
points and their motion vectors, respectively. 

so 9. A method of synthesizing an interframe predicted image according to any of claim 2 to 5. wherein 

the coordinates of pixels at the left upper end, right upper end, left lower end and right lower end of the image 
are (0, 0), (r, 0). (0, s) and (r, s) (where r and s are positive integers). 

the horizontal and vertical components of motion vectors at corner points (0. 0), (0. s) and (r. s) of the image 
55 take values of integral multiples or 1/n (where n is a positive integer). 

said method uses (uOO, vOO). (u02, v02) and (u03, v03) (where uOO. vOO. u02, v02. u03 and v03 are integers) 
which are n times as great as said motion vectors, and 

(uO. vO). (u1, v1). (u2, v2) and (u3, v3) which are k times as great as the horizontal and vertical components of 
motion vectors at points (i, j), (i+p, j), (i, j+q) and (i+p, j+q) are given by, 
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u'(*. y) = (uOO • r • s+<uO3-u02)x • s+(u02-u00)y • r)k ///(r • s • n) 
v*(x, y) = (vOO • r • s+(vO3-v02)x • s+(v02-vOO)y • r)k ///{r • s • n) 

uO=u'(i.D 

vO = v'(i. j) 

U1 = U*(i+P. D 

v1 = V(i+p. j) 

u2 = u'(i.j+q) 

v2 = v*(i. j+q) 
u3 = tT(i+p. j+q) 
v3 = v'(i+p. j+q) 

(where 7/r represents a division for rounding the quotient of the ordinary division into an adjacent integer when 
the quotient of the operation is not an integer, and its priority as an operator is the same as that of the ordinary 
multiplication and division) and 

further wherein three points among these four points and their motion vectors are used as representative 
points and their motion vectors, respectively. 

10. A method of synthesizing an interframe predicted image according to any of claims 2 to 5, wherein 

coordinates of pixels at the left upper end, right upper end. left lower end and right lower end of the image are 
(0, 0), (r, 0). (0. s) and (r, s) (where r and s are positive integers), 

the horizontal and vertical components of motion vectors at corner points (r, 0). (0, s) and (r. s) of the image 
take values of integral multiples of 1/n (where n is a positive integer), 

said method uses (u01. v01), (u02, v02) and (u03, v03) (where u01. v01, u02, v02. u03 and v03 are integers) 
which are n times as great as said motion vectors, and 

(uO. vO). (u1 , v1), (u2, v2) and (u3, v3) which are k times as great as the horizontal and vertical components of 
motion vectors at points (i, j"), (i+p, j). (i. j+q) and (i+p, j+q) are given by, 

u'(x, y) = (u03 • r • s+(u01 -u03)(r-x) • s+ (u02-u03)(s-y) • r)k///(r • s • n) 

V(x, y) = (v03 * r • s+(v01 -v03)(r-x) * s+ (v02-v03)(s-y) • r)k///(r • s • n) 

uO = u*(i. j) 

vO = v'(i. D 

u1 = u'(i+p. j) 

v1 = v'(i+p, j) 

u2 = u'(i, j+q) 

v2 = V(i. j+q) 

u3 = u'(i+p. j+q) 

v3 = v'(i+p, j+q) 

(where 7/r represents a division for rounding the quotient of the ordinary division into an adjacent integer when 
the quotient of the operation is not an integer, and its priority as an operator is the same as that of the ordinary 
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multiplication and division) and 

further wherein three points among these four points and their motion vectors are used as representative 
points and their motion vectors, respectively. 

5 1 1 . A method of synthesizing an interframe predicted image according to claim 6. wherein 

the coordinates of pixels at the left upper end. 

the coordinates of pixels at the left upper end, right upper end, left lower end and right lower end of the image 
are (0, 0), (r, 0). (0, s) and (r, s) (where r and s are positive integers), 
w the horizontal and vertical components of motion vectors at corner points (0. 0). (r, 0). (0, s) and (r, s) of the 

image take values of integral multiples of 1/n (where n is a positive integer), 

said method uses (uOO. vOO). (uOI. vOI), (u02, v02) and (u03, v03) (where uOO. vOO. u01. vOI. u02, v02, u03 
and v03 are integers) which are n times as great as said motion vectors, and 

(uO, vO), (u1 , v1), (u2 f v2) and (u3. v3) which are k times as great as the horizontal and vertical components of 
75 motion vectors at representative points (i. j). (i+p, j), (i, j+q) and (i+p, j+q) are given by. 

u'(x. y) = ((s-y)(uOO • (r-x)+u01 • x)+ y(u02 • (r-x)+u03 - x))k///(r • s • n) 

v'(*. y) = ((s-y)(v00 • (r-x)+v01 • x)+ y(v02 • (r-x)+v03 • x))k///(r - s • n) 

20 

uO = u'(i, j) 
vO=v'(i. j) 

25 U1 = u'(i+p. j) 

v1 = Vfl+p. j) 
u2 = u'(i. j+q) 

30 

v2 = V(i, j+q) 
u3 = u'(i+p. j+q) 

35 v3 = V(i+p, j+q) 

(where 7/T represents a division for rounding the quotient of the ordinary division into an adjacent integer when 
the quotient of the operation is not an integer, and its priority as an operator is the same as that of the ordinary 
multiplication and division). 

40 

1 2. A method of synthesising an interframe predicted image according to claim 1 , wherein the image is divided into two 
right triangles using a diagonal, and the global motion compensation is independently effected for the pixels con- 
tained therein based upon linear interpolation and/or extrapolation, wherein three representative points are used. 

45 13. A method of synthesizing an interframe predicted image according to any one of claims 2 to 11 , wherein m ir and/or 
7/r are further defined to represent an operation for rounding the quotient of an ordinary division into the most adja- 
cent integer. 

A method of synthesizing an interframe predicted image according to any of claims 2 to 1 1 . wherein 7f and/or 7/T 
are still further defined to represent an operation which, when the quotient of an ordinary division is a value of the 
sum of an integer and 1/2. rounds this value toward 0. 

A method of synthesizing an interframe predicted image according to any of claims 2 to 1 1 . wherein 7f and/or 7/T 
are still further defined to represent an operation which, when the quotient of an ordinary division is a value of the 
sum of an integer and 1/2. rounds this value away from 0. 

A method of synthesizing an interframe predicted image according to any of claims 2 to 1 1 . wherein 7/" is still fur- 
ther defined to represent an operation which, when the quotient of an ordinary division is a value of the sum of an 
integer and 1/2. rounds this value toward 0 when the dividend is negative and rounds this value away from 0 when 



16 

3NSOOCID: <EP 07973S7A2J_> 



EP0 797 357 A2 

the dividend is positive. 

17. A method of synthesizing an interframe predicted image according to any of claims 2 to 11 . wherein 7T is still fur- 
ther defined to represent an operation which, when the quotient of an ordinary division is a value of the sum of an 

s integer and 1/2, rounds this value away from 0 when the dividend is negative and rounds this value toward 0 when 

the dividend is positive. 

18. A method according to claim 1, wherein when r denotes the lateral length of the image and s denotes the vertical 
length of the image, p is smaller than r, 2p is larger than r t q is smaller than s, and 2q is larger than s. 

10 

19. A method according to claim 1, wherein when r denotes the lateral length of the image and s denotes the vertical 
length of the image, p/2 is smaller than r, p is larger than r, q/2 is smaller than s, and q is larger than s. 

20. A method of coding and decoding an image based upon the method of claim 1 . 

is 

21 . A method of coding an image based upon the method of claim 1 , wherein the data related to the motion vectors of 
the representative points are directly coded. 

22. A method of decoding an image based upon the method of claim 1 , wherein the data related to the motion vectors 
20 of the representative points that have been directly coded as coded data are used by reproducing the data. 

23. A method of coding an image based upon the method of claim 1 , wherein the data related to motion vectors of 
points at corners of an image are directly coded. 

25 24. A method of decoding an image based upon the method of claim 1. wherein the data related to motion vectors of 
points at corners of an image that have been directly coded as coded data, are used by reproducing the data. 

25. A storage medium for storing a global motion compensation program, the program comprising: 

30 calculating the motion vectors for all pixels in an image by effecting linear interpolation and/or extrapolation of 

the motion vectors at three or four representative points for the image, and 

assuming sampling intervals of the pixels to be 1 in both horizontal and vertical directions with respect to the 
image, and assuming sampling points for which horizontal and vertical components of coordinates of the sam- 
pling points are both integers, 

35 wherein the representative points are three or four of the points having the coordinates (i, j), (i+p. j). 0. j*<0. and 

(i+p f j+q) (where i and j are integers, and p and q are positive integers), and wherein p and q are 2 a and 2 P (a 
and p are positive integers), respectively. 
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